1. Technical Field
The present disclosure generally relates to processing of an image, and more particularly, to an information processing apparatus, an information processing system, an image processing method, and computer-readable storage medium storing a program that restores an image.
2. Description of the Related Art
Images photographed by digital cameras may be generally degraded due to mixing with image-degrading noise, such as sensor noise. Therefore, in some instances, denoising processing may be required. In other aspects, a photographed image may be a degraded image if the image has a low resolution or has blurring generated therein. In some instances, super resolution processing for enhancing the resolution of the image or deblurring processing for reducing the blurring may be required.
A common image restoration method used for restoring an image that is degraded due to mixing with noise (hereinafter, represented as “degraded image”) may include calculating a difference between a denoised image (hereinafter, represented as “restored image”) and the degraded image. Further, the common image restoration method may include calculating the extent of smoothness of the entire restored image. Further, the common image restoration method may include creating a restored image, which is an image for which the calculated difference is small, and the entire restored image is smooth.
In some aspects, the common image restoration method may use an objective function in which a term for smoothing an image (hereinafter, represented as “regularization term”) is added to a term represented using the sum of squares of a difference between a degraded image and a restored image (hereinafter, represented as “data term”). Further, the common image restoration method may determine (create) the restored image such that the objective function can be minimized.
As a regularization term, total variation (TV) regularization may be widely used. The TV regularization may assume that an image includes a region with a small change in pixel value between neighboring pixels and a region with a large change in pixel value between neighboring pixels (region in which sharp change occurs). Further, based on that assumption, the TV regularization may apply a penalty depending on the L1 distance between high-frequency components to the restored image. The “L1 distance” may be the sum of absolute differences between vector components where pixels are represented as vectors.
The technology described in a related art document may use linear filter processing to minimize the objective function, including the TV regularization term. The technology may create a denoised restored image based on the above-mentioned method. This method may be called DTVF (Digital TV Filter).
The DTVF will be further described with reference to the drawing.
FIG. 7 is a block diagram illustrating an example of the configuration of an information processing apparatus 90 which may use the DTVF.
As illustrated in FIG. 7, the information processing apparatus 90 that uses the DTVF may include a variation calculation unit 910, a filter coefficient calculation unit 920, and a filter processing unit 930. The variation calculation unit 910 may calculate a local variation based on values of differences between the luminance of a focused pixel and the luminance of surrounding pixels of the focused pixel. Based on the calculated local variation, the filter coefficient calculation unit 920 may calculate a filter coefficient matrix for suppressing a variation between neighboring pixels. The filter processing unit 930 may perform filter processing of an image using the calculated filter coefficient matrix. The variation calculation unit 910 may calculate a local variation in the filtered image. The information processing apparatus 90 may iterate this processing to create a restored image (output image). The information processing apparatus 90 using the DTVF may denoise the image by using the above-mentioned iterative processing. In this manner, the processing using the simple nonlinear filter may be configured. Accordingly, the denoising processing described in the related art document generally may have a low calculation cost.
However, the technology described in the related art document may be a method specializing in denoising. Accordingly, the technology cannot be utilized to image restoration processing different from the denoising super resolution processing and deblurring processing, for example.
Specifically, this limitation may be present because of the following reasons. The nonlinear filter used in the related art document may be a filter that is designed to minimize the sum of squares of differences between a restored image and a degraded image. This may be generally because noise is added on a random basis. However, small differences between a restored image and a degraded image may not be necessarily preferable. For example, differences between a degraded image and a restored image deblurred by use of deblurring processing may be generally large. For this reason, the technology described in the related art document may not be applicable to image restoration processing such as deblurring processing.
In other aspects, the technology described in the related art document may have a problem of being unusable for the degraded-image restoration processing other than the denoising.
Exemplary embodiments of the present disclosure may solve one or more of the above-noted problems. For example, the exemplary embodiments may provide an information processing technique for creating a restored image supported by denoising and other degraded-image restoration.